The following documents are incorporated herein by reference:    1. Decker, M. L., R. R. Kay, N. G. Rackley, “Multispectral Thermal Imager (MTI) satellite hardware status, tasking and operations,” Proc. Spie, vol. 4381, pp. 184-94, 2001.    2. Fonseca, L. M. G., and B. S. Manjunath, “Registration techniques for multisensor remotely sensed imagery,” Journal of Photogrammetry Engineering and Remote Sensing, vol. 62 (9), p. 1049-1056, September 1996.    3. Brown, L. G. “A survey of image registration techniques,” ACM Computer Survey, vol. 24, no. 4, pp. 325-376, 1992.    4. Kuglin, C. D., and D. C. Hines, “The phase correlation image alignment method,” in Proc. Int. Conf. Cybernetics Society, pp. 163-165, 1975.    5. Smith, J. L., S. E. Momomatsu, J. G. Taylor, K. J. Jefferson, and B. R. Stallard, “Semi-autonomous registration of satellite imagery using feature fitting,” in Proc. SPIE, vol. 4381, pp. 447-454, 2001.    6. Shekarforoush, H., M. Berthod, and J. Zerubia. “Extension of phase correlation to subpixel registration,” IEEE Trans. On Image Processing, vol. 11, no. 3, pp. 188-200, March 2002.    7. Pope, P., J. Theiler, and A. Galbraith. “LANL experience with coregistration of MTI imagery,” in Proc. SPIE, vol. 5159, pp. 139-146, 2003.    8. Hsieh, J. W., Liao, H. Y., Fan, K. C., “Image Registration Using a New Edge-Based Approach”, Computer Vision and Image Understanding, vol. 67, no. 2, pp. 112-130, August 1997.    9. Van der Steen, A. J., “An Evaluation of Some Beowulf Clusters,” Cluster Computer, vol. 6, issue 4, pp. 287-297, October 2003.    10. Goodman, J. W. Introduction to Fourier Optics, p. 9, New York, McGraw-Hill, 1968.
The Multispectral Thermal Imager Satellite (MTI) is a multispectral pushbroom system that acquires 15 unique spectral bands of data from 0.45-10.7 microns, with resolutions of 5 m for the visible bands and 20 m for the infrared. Scene data are collected on three separate sensor chip assemblies (SCAs) mounted on the focal plane. The MTI satellite was launched on a Taurus rocket into a 1 pm sun-synchronous 595 km orbit on Mar. 12, 2000. Four of the bands are visible near-infrared (VNIR) bands with a 5 m ground sampling distance (GSD), seven are medium-wave infrared (MWIR) with a 20 m GSD, and five are thermal infrared (TIR) with a 20 m GSD. FIG. 1 shows the 15 unique spectral bands.
The MTI focal plane is distributed among three identical sensor chip assemblies (SCAs) shown in FIG. 2. The left and right SCAs are rotated 180 degrees with respect to the central SCA so that the VNIR bands for each SCA (see I) are located closest to the optical axis. The optical axis extends perpendicularly to the focal plane, i.e. perpendicularly into the page of FIG. 2, as shown at 21. Each identical SCA contains 832 VNIR pixels and 208 IR pixels in the cross-track direction, which cross-track direction is shown at 23. The inner edges of the left and right SCAs (SCAs 2 and 3) share a small pixel overlap region with central SCA 1. Detector rows for each band are arranged sequentially by order of increasing wavelength and extend away from the optical axis in the along-track direction (i.e., parallel to the scan direction). In FIG. 2, the VNIR bands are located in section I, the MWIR bands in section II, and the TIR bands in section III. Images are created with MTI by sequentially activating detector bands across the ground target, starting with the longest-wavelength band (N) in SCA 1, and ending with band N on SCAs 2 and 3. According to conventional pushbroom operation, each individual row of detectors (corresponding to one of the spectral bands) is activated at a plurality of points in time while moving over the target, thereby producing an image defined by a two dimensional array of pixel data.
The unique arrangement of the MTI focal plane results in 45 separate images (15 bands by 3 SCAs) being delivered for each image collect. These band images are displaced with respect to each other because of the physical difference in their positions on the focal plane and because of spacecraft motion and attitude changes during the (approximately 4 second long) image collections process. To combine these 45 images into a single coregistered MTI image cube (referred to as a level 1R product), both interband misregistration within each SCA and the misregistration between the SCAs themselves must be corrected.
Image registration is a fundamental issue for both image processing and image analysis. Images requiring registration can exhibit temporal changes, viewpoint differences, or unknown scene overlap, and each of these relationships presents a unique registration challenge. With respect to multispectral and hyperspectral sensors, accurate interband registration is critical in order to analyze the spectral nature of the data. Many image registration techniques and the geometric transformations they correct are discussed in the literature. Fonseca outlines the registration problem in four steps: feature identification, feature matching, spatial transformation, and interpolation if necessary (document No. 2 above). Registration techniques can be grouped into the categories of area-based methods and feature-based methods (document No. 3 above). Feature-based methods typically utilize one technique to locate spatially distinct features and another technique to align the bands with respect to those features, thus registering the image. Area-based methods combine feature identification and feature matching into a single step through correlation, but the importance of quality scene features in the image is still of paramount importance.
Phase correlation is an area-based method well suited to detecting translational shift errors between imagery (document No. 4 above). As area-based techniques, most correlations require that the images being registered share a large region of overlap to produce good results. A shifting window is typically used to correlate one small area of a reference image within a larger windowed area of a target image to figure out the point of best registration (document No. 5 above). While images from similar wavelengths tend to look alike, images from different wavelengths can exhibit vastly different features. Several effects make conventional correction between such widely separated spectral bands very difficult. Examples of such effects are contrast reversals between the widely separated bands, and the loss of contrast in respectively different portions of the respective images produced by the widely separated bands. Normalized cross correlation is an area technique traditionally used for translation registration of images from similar wavelengths (document No. 3 above). Cross correlation is dependent only on statistical similarities between pixel intensities, so it does not work well between images from different wavelengths. Phase correlation has several desirable properties such as uniform variations, gain offsets, and constant mean shifts that make it robust against the effects that mislead other correlation techniques (document No. 6 above).
Generation of properly registered MTI image requires two separate registration steps. First, interband registration must be performed on the 15-band cube generated by each of the three SCAs. Second, the registered image cubes from each SCA must be mosaicked together (in the overlapping border regions) to produce the final registered MTI image cube. Several registration techniques have already been developed for MTI imagery, each tailored to a different kind of image product. Los Alamos National Laboratory has developed multiple techniques, among them an automated model-based photogrammetric approach that involves “tweaking” by cross-correlation, (document No. 7 above). Sandia National Laboratories has also developed several registration techniques, including a feature-fitting method (document No. 5 above). In order to guarantee registration accuracy, Sandia currently operates a manual registration tool used in the Sandia Image Processing and Exploitation (SIPEX) image production pipeline for MTI imagery. This manual registration tool is called the Package for Registering Images from MTI (PRISM).
The PRISM tool allows translation registration by manually shifting one MTI band as referenced to another MTI band. The analyst finalizes registration shifts for each band when all image pairs overlay as closely as possible in a given region of an image. These integer-pixel shifts are used in creating the final un-resampled registered image product. Manual registration, however, is a time-consuming process and becomes very difficult when no very distinct features exist in the image. Human time requirements would be reduced significantly if images were automatically registered, and analysts only needed to correct occasional misregistrations in a quality-control step. It is therefore desirable to provide a generalized and robust way to automatically register images, such as images in the MTI SIPEX pipeline.
Exemplary embodiments of the invention use phase correlation with enhancements to achieve robustness. Multispectral images are first gradient-filtered on a band-by-band basis using an edge detection method. As feature edges are typically preserved throughout all bands of a multispectral image, edge-filtering the input images provides robustness over wide wavelength ranges where contrast inversions and other wavelength-dependent intensity changes are common. These gradient images are then phase-correlated, resulting in a correlation surface. The correlation surface can be edge-filtered in a single spatial direction to remove certain correlation effects as needed. The final shift values for the image pair are obtained based on the edge-filtered correlation.